Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users

Recovering 3D human body motion from video is important for applications such as rehabilitation assessment and sports performance evaluation. For prosthesis users, however, this requires capturing both the natural body joints and the geometry of the prosthetic device, which existing methods are not designed to address. Model-based estimators rely on body models trained on non-amputee individuals and cannot represent prosthesis geometry, while model-free reconstruction methods lack body kinematic priors and are unreliable under occlusion. Neither approach alone can address this task, and the lack of suitable data has prevented any systematic study.

This challenge is particularly prominent for users of running-specific prostheses (RSPs), where the RSP has a complex curved geometry and moves dynamically during exercise. To fill this gap, we collect RSP3D, the first 3D dataset of RSP users, covering essential daily-life and exercise actions from participants with diverse amputation conditions, using a multi-camera setup with marker-based motion capture. We formally define the task of prosthesis-aware 3D pose estimation and evaluate representative model-based and model-free methods in a zero-shot setting, confirming their individual limitations. Based on these findings, we propose a hybrid baseline that combines model-based body joint estimation with model-free RSP shape recovery, establishing a starting point for future research on this task.